Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

separate infrastructure environments #1074

Merged
merged 3 commits into from
Dec 22, 2022
Merged

separate infrastructure environments #1074

merged 3 commits into from
Dec 22, 2022

Conversation

afeld
Copy link
Contributor

@afeld afeld commented Oct 20, 2022 •
edited
Loading

Closes #830. Running list of TODOs:

  • Update docs around environments, removing references to "slots"

  • Update deployment pipeline

    • Ensure it's connecting to the right subscription/backend

  • DevSecOps:

    • Recover or purge Key Vault KV-CDT-PUB-CALITP-D-001

      • Our permissions are limited on Key Vaults so have been trying to work around that

    • Create VNet and Subnet(s) for dev and test

      • Provide VNet and Subnet names, and the Resource Group they're in

      • Give read permissions:

        Error: retrieving Subnet: (Name "SNET-CDT-PUB-CALITP-D-001" / Virtual Network Name "VNET-CDT-PUB-SHRD-W-D-001" / Resource Group "RG-CDT-PUB-SHRD-W-D-001"): network.SubnetsClient#Get: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="AuthorizationFailed" Message="The client 'live.com#[email protected]' with object id [REDACTED] does not have authorization to perform action 'Microsoft.Network/virtualNetworks/subnets/read' over scope '/subscriptions/[REDACTED]/resourceGroups/RG-CDT-PUB-SHRD-W-D-001/providers/Microsoft.Network/virtualNetworks/VNET-CDT-PUB-SHRD-W-D-001/subnets/SNET-CDT-PUB-CALITP-D-001' or the scope is invalid.

    • Create Test Resource Group

    • Create Azure DevOps Service Connection to Development Subscription

    • Configure Front Door

    • Clean up unused resources in Development Resource Group:

      • Storage account sacalitpd001
      • App Service Plan ASP-RGCDTPUBVIPCALITPD001-9628

    • Provide Active Directory Group ID

  • Rename resources to:

    • Prevent them from being destroyed and recreated
    • Make the plan output shorter and thus easier to review

  • Import test resources

  • Update DNS

  • Set up webhooks from GitHub

  • Set configuration variables and files for new environments

  • downgrade App Service plan #1142

  • Update IP address ranges in Eligibility Server

@github-actions github-actions bot added the infrastructure Terraform, Azure, etc. label Oct 20, 2022
afeld
afeld commented Oct 20, 2022
View reviewed changes
terraform/main.tf Outdated
@@ -4,7 +4,7 @@ terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "~> 3.25.0"
version = ">= 3.0.0, < 4.0.0"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The version gets locked anyway, so no need to have a tighter constraint.

@afeld afeld force-pushed the chore/separate-environments branch from 60834c9 to d745102 Compare October 20, 2022 16:28
afeld
afeld commented Oct 20, 2022
View reviewed changes
terraform/main.tf

data "azurerm_resource_group" "prod" {
name = "RG-CDT-PUB-VIP-CALITP-P-001"
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to environment.tf.

@afeld afeld force-pushed the chore/separate-environments branch 11 times, most recently from 14231da to 8a2f169 Compare October 26, 2022 15:33
This was referenced Oct 26, 2022
Merged
Merged
Merged
@afeld afeld force-pushed the chore/separate-environments branch 3 times, most recently from e44e05c to 2e81b11 Compare October 27, 2022 05:36
@afeld afeld force-pushed the chore/separate-environments branch 2 times, most recently from 03c8ee2 to 156c2e1 Compare November 18, 2022 20:52
This was referenced Nov 22, 2022
Merged
Merged
afeld
afeld commented Nov 22, 2022
View reviewed changes
terraform/network.tf
name = "SNET-CDT-PUB-CALITP-P-001"
virtual_network_name = "VNET-CDT-PUB-SHRD-W-P-001"
resource_group_name = local.network_resource_group_name
subnet_id = "/subscriptions/${data.azurerm_client_config.current.subscription_id}/resourceGroups/${local.network_resource_group_name}/providers/Microsoft.Network/virtualNetworks/${local.vnet_name}/subnets/${local.subnet_name}"
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removes the need to have read access to the subnet.

@afeld afeld force-pushed the chore/separate-environments branch from e465d7c to 9eb4847 Compare November 22, 2022 06:56
@afeld afeld marked this pull request as ready for review November 22, 2022 06:59
@afeld afeld requested a review from a team as a code owner November 22, 2022 06:59
@afeld afeld force-pushed the chore/separate-environments branch from 6348566 to ce7034d Compare November 22, 2022 07:06
@afeld afeld force-pushed the chore/separate-environments branch from 065325b to d3f4e75 Compare December 6, 2022 04:14
@afeld
Copy link
Contributor Author

afeld commented Dec 12, 2022

The Terraform here is all ready to go, thus changes can be deployed manually (with terraform apply). The Pipeline changes are on hold pending #830 (comment).

@afeld afeld force-pushed the chore/separate-environments branch from d3f4e75 to 1072018 Compare December 12, 2022 22:23
@afeld afeld force-pushed the chore/separate-environments branch 3 times, most recently from f36c00e to 2ca301a Compare December 20, 2022 17:26
@afeld afeld mentioned this pull request Dec 20, 2022
Closed
@afeld afeld force-pushed the chore/separate-environments branch 7 times, most recently from 66868eb to df68eea Compare December 21, 2022 21:44
@afeld
chore(Terraform): deploy isolated environments
af86828 
Instead of deploying to different slots in a single Resource Group, use separate ones to fully isolate environments. This includes Pipeline changes based on

https://github.com/cal-itp/eligibility-server/tree/7be712310fedc7c0459d7baa36f632f5a1dee765/terraform/pipeline
@afeld afeld force-pushed the chore/separate-environments branch from df68eea to c4748ef Compare December 21, 2022 21:50
@afeld
Copy link
Contributor Author

afeld commented Dec 21, 2022

We are now seeing the pipeline working appropriately; see the run against dev and the run against prod, each of which is using the appropriate service connection, workspace, etc.

@afeld afeld mentioned this pull request Dec 21, 2022
Closed
@afeld afeld force-pushed the chore/separate-environments branch from c4748ef to 0610720 Compare December 21, 2022 22:39
@afeld afeld marked this pull request as ready for review December 21, 2022 22:53
thekaveman
thekaveman reviewed Dec 21, 2022
View reviewed changes
terraform/network.tf
# VNet uses a shared Resource Group, different from App Service Resource Group we use for Benefits stuff
network_resource_group_name = local.is_prod ? "RG-CDT-PUB-SHRD-W-P-001" : "RG-CDT-PUB-D-001"
vnet_name = local.is_prod ? "VNET-CDT-PUB-SHRD-W-P-001" : "VNET-CDT-PUB-D-001"
subnet_name = local.is_prod ? "SNET-CDT-PUB-CALITP-P-001" : "SN-CDT-PUB-CALITP-${local.env_letter}-001"
Copy link
Member

@thekaveman thekaveman Dec 21, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the right side of this ternary correct? "SN-CDT-PUB-CALITP-${local.env_letter}-001"

Should it be "SNET-CDT-PUB-CALITP-${local.env_letter}-001"? (SNET instead of SN)

Copy link
Contributor Author

@afeld afeld Dec 22, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It matches what @cal-itp/cdt-devsecops named it, and I/we don't have permissions to modify 🤷

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As part of #1077, maybe we can raise where we are seeing inconsistencies.

terraform/pipeline/workspace.py Show resolved Hide resolved
terraform/pipeline/workspace.py Outdated Show resolved Hide resolved
@afeld
fix(pipeline): handle the pipeline being manually triggered
04a2d7e 
This is the case when the `Run` button is pressed in the UI. Copied from cal-itp/eligibility-server@a9edf47.
@afeld afeld merged commit e7683cc into dev Dec 22, 2022
@afeld afeld deleted the chore/separate-environments branch December 22, 2022 15:06
@afeld
Copy link
Contributor Author

afeld commented Dec 22, 2022

I had already run a terraform apply locally with these changes, so No changes from the Pipeline's apply on dev. Expect to see changes as we merge that up to test, and then the merge to prod will be the big one that deletes the slots, etc.

@angela-tran angela-tran mentioned this pull request Apr 26, 2023
Closed
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@angela-tran angela-tran angela-tran approved these changes

@thekaveman thekaveman Awaiting requested review from thekaveman

Assignees
No one assigned
Labels
infrastructure Terraform, Azure, etc.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

have a way to test Terraform changes more safely
3 participants
@afeld @thekaveman @angela-tran